Release Notes for Q1 2023
Explore the new features added in this update!
Updated in: April 2023
Release Version: 0.13
| Feature | Description |
|---|---|
| Azure Key Vault support in Calibo AccelerateData Pipeline Studio. |
Calibo Accelerate Data Pipeline Studio now supports Azure Key Vault as an identity provider to store credentials for the following tools:
|
| Support to upload dependent JAR files for AWS Glue Integration and Transformation jobs. | Calibo Accelerate Data Pipeline Studio now includes a feature for uploading dependent JAR files that are necessary for AWS Glue to execute integration and transformation jobs from the Calibo Accelerate platform. Users have the option to choose whether to have the files uploaded by the platform or to do it themselves using a toggle switch. A list of the required files is provided along with a link to download them, and they can be uploaded to a specified S3 location. |
| Support to change configuration of newly added stages of a promoted pipeline. | When a pipeline is moved from the Dev stage to the next stage, and then modifications are made to the pipeline in the Dev stage, the updated version of the pipeline can now be promoted to the next stage without affecting the configuration of existing nodes. This means that any changes made to newly added nodes can be made without impacting the configuration of the nodes that were already present in the pipeline. |
| IAM roles added in AWS Glue integration job for restrictive access. |
The Calibo Accelerate Data Pipeline Studio has introduced a new feature that allows users to provide details of an existing or new IAM role to provide granular permissions for AWS Glue Integration job runs. |
| Support for creating recipe jobs and profile jobs with data quality rules for AWS Glue DataBrew. | With the latest release, Calibo Accelerate Data Pipeline Studio has added the capability to create recipe jobs or profile jobs for AWS Glue DataBrew, using new or existing datasets and rule sets for data quality. Additionally, the platform now offers support for Orchestrator Agent in AWS Glue DataBrew. |
| Orchestrator agent support extended to data integration, data transformation, and data lake stages using Snowflake. | Calibo Accelerate Data Pipeline Studio now offers support for Orchestrator Agent in the data integration, data transformation, and data lake stages using Snowflake. |
| Node-level notifications are now supported for SQS and SNS along with pipeline-level notification. |
The platform now allows enabling notifications for nodes in a pipeline using SQS and SNS. This feature is currently supported for following tools:
|
| Support for assigning permissions to objects in Snowflake warehouse. | With the latest update, Calibo Accelerate Data Pipeline Studio now allows users to assign permissions to objects in their Snowflake warehouse using two options: Map existing role or Create New Role. Using the Map Existing Role option, users can assign all permissions available in an existing role in Snowflake to a user in Calibo Accelerate. Alternatively, users can Create a new role directly from the Calibo Accelerate platform and assign the required permissions to it. Once created, the new role is added to Snowflake, allowing for efficient management of permissions across platforms. |
| Low code data transformation now supported in AWS Glue. | A templatized AWS Glue Transformation job has been added to Calibo Accelerate Data Pipeline Studio, enabling the use of join, union, and aggregate functions. |
| Calibo Accelerate Orchestrator Agent support extended for AWS Kinesis streams for data source stage. | Calibo Accelerate Data Pipeline Studio now allows users to set up AWS Kinesis as a data source by using the Calibo Accelerate Orchestrator Agent. |
| Delta file support is now available for Amazon S3 source and target nodes. | Calibo Accelerate Data Pipeline Studio now offers delta file support for both Amazon S3 source and target nodes through Databricks. This functionality allows users to select a specific timestamp to read a particular version of data from an S3 bucket or append data to an S3 bucket for incremental data load. |
| Calibo Accelerate Orchestrator Agent support is extended to some tools in Data Pipeline Studio. |
The Calibo Accelerate Orchestrator Agent helps you establish a secure connection between the tools deployed in your private network and the Calibo Accelerate platform. Thus, you can access the tools from within the Calibo Accelerate platform without sharing any credentials with the platform. The Calibo Accelerate Orchestrator Agent support is now available in Data Pipeline Studio for the following tools:
|
| Revamped Kubernetes Cluster Configuration screen | The Kubernetes Cluster Configuration screen UI has been revamped with more options to configure the Kubernetes Cluster. Moreover, Calibo Accelerate Orchestrator Agent support is also available now for Kubernetes. |
| New version of Calibo Accelerate Orchestrator Agent available | The Calibo Accelerate Orchestrator Agent is updated to a new version. On the Calibo Accelerate Orchestrator Agents screen, click Update Available for instructions to upgrade the agent to the latest available version. See Upgrading Calibo Accelerate Orchestrator Agent. |
| Deactivating a team is now supported in the Calibo Accelerate platform. | You can deactivate a team by selecting the Deactivate option and specifying the month from which the team is to be deactivated. Before you confirm the deactivation, you can click View Details to view a list of products that the team is associated with, and also a list of users that are part of the team. |
| Azure Key Vault support is now available in the Calibo Accelerate platform. |
The Calibo Accelerate platform now supports secrets management tool Azure Key Vault to store secrets for the following tools:
|
| Calibo Accelerate Orchestrator Agent support is now available for Microsoft Azure. |
The Calibo Accelerate Orchestrator Agent support is now available for the following tools and technologies deployed in a Microsoft Azure account:
The Calibo Accelerate Orchestrator Agent helps you establish a secure connection between the tools deployed in your private network and the Calibo Accelerate platform. Thus, you can access the tools from within the Calibo Accelerate platform without sharing any credentials with the platform. |